An integrated system for cooperative man-machine interaction

An Integrated System for Cooperative Man-Machine Interaction C. Bauckhage, G.A. Fink, J. Fritsch, F. Kummert, F. L omker, G. Sagerer, S. Wachsmuth Technical Faculty, Bielefeld University P.O. Box 33594, Bielefeld, Germany http://www.techfak.uni-bielefeld.de/techfak/ags/ai/ Abstract To establish robotic applications in human environments as e.g. o ces or private homes the robotic systems must be instructable by ordinary users in a natural way. In interpersonal communication humans usually apply di erent sensory information and are capable of integrating all perceptual cues fast and consistently. Additionally, knowledge acquired during the communication process is directly used to resolve ambiguities. As a step towards realizing similar capabilities in automatic devices this paper presents an integrated system combining automatic speech processing and image understanding. The system is intended to be an intelligent interface of a robot which manipulates objects in its surroundings according to the instructions of a human. The enhanced capabilities necessary for carrying out a multimodal man-machine dialog are realized by combining statistical and declarative methods for inference and knowledge representation. The e ectiveness of this approach is demonstrated using an examplary dialog from our construction task domain.